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Dear Sir: 

In response to the Notice of non-complaint appeal brief dated June 4, 2007, 
Applicant resubmits the appeal brief, with an indication that all of claims 1-20 are being 
appealed. 

This Appeal Brief is being filed under 37 CFR 41 .37, thereby perfecting the 
Notice of Appeal which was originally filed on February 21 , 2007. 
The Appeal Brief fee was previously paid (small entity). 
The Appeal Brief sections required by Rule 41 .37 follow. 
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Real Party in Interest 

The application is not assigned, and hence the inventor Scott C. Harris is the real 
party in interest. 

Related Appeals and Interferences 

There are no known related appeals and/or interferences, 

Status of Claims 

Claims 1-20 are pending in the case. Each of these claims have been rejected 
over prior art. Each of claims 1-20 are appealed herein. 

Status of Amendments 

No amendment was filed subsequent to the final rejection dated November 22, 

2006. 

Summary of Claimed Subject Matter 

Claim 1 requires a client that allows entering image information. See page 4, 
last five sentences (paragraph 22 of the specification). 

Claim 1 requires a server that stores image information as a database of 
information of an image. See page 4 of the specification, lines 10-16 (paragraph 18). 
The server uses the image information to search a database "for items to be purchased 
which meet criteria specified in said image information and forming search results 
based on said image information". Page 6, lines 9-13 describe searching the database 
in this way. 
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Claim 10 requires entering image information to a client on the network, see 
page 4 last five sentences. Claim 10 further requires sending the information to a 
server and using the image information to search database information on the server, 
see page 6 lines 9-13. Claim 10 requires that search results from the server are 
returned, see generally page 2 lines 4-14, and page 3 of the specification last four lines. 

Claim 17 requires a computer that includes a database with image information 
and price information for those images. See page 3 last four lines. Claim 17 also 
requires the computer except search information over the network see page 4 lines 17- 
20 and 1-5. Finally, claim 17 recites returning search results; See page 2 lines 4-14 
and last four lines of page 3 of the specification. 

Grounds of Rejection to be Reviewed on Appeal 

Are claims 1-20 properly rejected under 35 U.S.C. 103(a) as allegedly being 
unpatentable over Crill in view of Hess. 

Is claim 4 properly rejected under 35 U.S.C. 103(a) as allegedly being 
unpatentable over Crill in view of Hess and further in view of Jain. 

Argument 

No amendment was filed subsequent to the November 22, 2006 official action. 
Note that there was a prior Pre-Appeal Brief request for review, which resulted in 
reopening of prosecution. The same issue is being reraised even after that Pre-Appeal 
brief resulted in the case being returned to the Examiner. Hence, the undersigned 
believes that further dialogue outside the appeals process will not be useful. Hence, 
this appeal brief is filed to address the existing rejections in the case. 
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Shopping on the Internet has become ubiquitous. What if, however, you want to 
buy a product and you don't know what the name of that product is? A perfect 
example, and the one given in the specification, is the item shown in Figure 3 of the 
application. What if you want one of those things? This item is difficult to describe in 
text or words, but easily describable visually. 

The present application recognizes this problem - and the claims defines the 
ability to carry out various functions related to purchasing of items, even when you don't 
know words relating to that item, or when searching by visuals is preferable to 
searching by words. The present system therefore defines a visual database that 
allows this. Nothing in the prior art has noticed this problem, and nothing in the prior art 
has suggested the claimed solution. The rejection cites patents that show technology 
that could be used IF there was any motivation to carry out the claimed subject matter - 
however there is no suggestion to carry out this subject matter in the cited prior art . 
There is no teaching or suggestion of the 'source of the problem'. Failure of the prior 
art to identify the source of a problem shows unobviousness. Eibel Process v 
Minnesota & Ont. Paper , 261 US 45 (1923). 

Claims 1-20 stand rejected under 35 U.S.C. 103(a) as allegedly being 
unpatentable over Crill in view of Hess. With all due respect, this contention is 
respectfully traversed, the rejection provides no suggestion of the problem or the 
solution, and does not meet the Patent Office's burden of providing a prima facie 
showing of unpatentability. 

Initially, applicants readily admit that the concept of searching an image 
database for matches is well-known in the art. In fact, if this was not well-known in the 
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art, the present specification would have to disclose the science of precisely how to 
search image databases for matches. Paragraph 1 6 of the present specification 
explains that existing search techniques are used to allow searching a database of 
image information to obtain images of items for sale, based on the image matching. 

While other aspects are claimed also, this basic feature is nowhere taught or 
suggested by the cited prior art. The prior art does teach various features that could be 
used as technology that could carry out the claimed subject matter, but the prior art 
does not actually teach or suggest using that technology in the way claimed. The prior 
art further does not disclose any reason why one might want to use that technology in 
the way claimed - that is, the prior art does not disclose the 
source of the problem being addressed by the claims. 

Initially, Crill teaches a system that allows storing things on a computer and 
searching them. One of the aspects of Crill is that an image database can be searched 
for images. Again, the present specification readily acknowledges that the prior art 
taught searching image databases for images. 

Crill teaches that this search method can be used by people "looking for a 
specific image or who are attempting to find matches between an image... and images 
in the database". See column 1 lines 34-37 of Crill. Crill says not one word about using 
this image database searching technique for finding items to be purchased. Each of 
the claims on appeal include the limitation of using their image database searching 
technique for finding items to be purchased. In fact, the official action dated November 
22, 2006, on page 3, admits that Crill "fails to teach a sever for items to be purchased" 
(sic). 
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The rejection alleges, however, that this is well-known in the art and would have 
been obvious based on Hess. With all due respect, this combination is based on 
hindsight, and the rejection itself is enabled by the teaching of the present application. 
There is not one word within either of the two references that suggests their 
combination in the way that is now contemplated by the rejection. 

Hess teaches an online trading environment, and shows the database searches 
can return results. The database searches which are suggested by Hess, however, are 
conventional queries. Hess teaches that a user provides a textual description and 
terms for an item. See column 3 lines 27-29. Once the item is associated in this way, 
thumbnail images can be formed, see column 3 lines 38-47. In other words, Hess does 
produce a database. Hess includes extensive discussion of the images associated with 
that database. So Hess has images in his database. There is not one word in Hess, 
however, about searching through those images using an image search technique . In 
fact, the searching which is done by Hess is conventional text based searching. 
Nothing in Hess suggests that anything other than text based searching should be 
used. So where does one obtain motivation to search through the extensive images 
that are obviously input in the database contemplated by the Hess patent? Certainly 
not from Grill. Crill shows that the science for searching image databases is well- 
known, but never suggests that image databases should be searched looking for items 
for sale. In fact, the motivation to search through Hess' images comes from the present 
specification . 

The law on the subject is clear. The Patent Office must show reasons why it a 
skilled artisan, confronted with the same problems with no knowledge of the claimed 
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invention, would select the elements from the prior art references for combination in the 
matter claimed. In re Rouffet 149 F 3d 1350, (Fed Cir 1998). The Patent Office may 
take into account knowledge of those having ordinary skill in the art, and common 
sense. See for example in re Dance , 1 60 3d 1 339 (Fed Cir 1 998). However, one 
cannot use hindsight in making the combination. 

The factual situation in this case demonstrates that hindsight was used in 
attempting to contend that these claims are obvious. The tools to do image database 
searching are available: Crill shows that a person having ordinary skill in the art would 
have understood that image database searching was available. Hess shows searching 
a database that has information about images therein. The reality is that it simply never 
occurred to anyone that you would obtain advantages for being able to search the 
image database. 

One reason why it did not occur to anyone is - the problem was not suggested 
or obvious from the prior art. Therefore, since the problem was not suggested, it could 
not by any stretch of the imagination be obvious to suggest techiques to solve that 
problem. The understanding that this would provide advantages, in essence, the 
source of the problem, comes from the present specification. Simply putting these two 
references together does not teach a person of ordinary skill in the art that there is even 
a problem therein. 

A person having ordinary skill in the art would be presumed to read these 
references and not even find any motivation in them to use Crill as an image searching 
technique in order to search the images in Hess. The tactic, by itself, is completely 
unobvious. 
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The rejection says that Crill fails to teach a server for items to be purchased but 
states that these "limitations are well-known in the art". The point is, however, that Crill 
provides no teaching that would suggest to a person of ordinary skill in the art that there 
would be any advantages or use from searching through the images to find images of 
items for sale. Similarly, Hess shows images in the database, it shows thumbnails and 
returns search results, but says not one word about searching the image database -- 
even though technology for doing so is clearly available. The contention that it would 
have been obvious to combine these references "in order to allow prospective 
purchasers to make a more informed decision by providing an improved user interface 
for online commerce sites" (sic) is quite simply based on hindsight, since the source of 
the problem is never revealed in either Crill or in Hess. 

The claimed subject matter is about searching for items to buy. The "informed 
decision" relied on by the official action does not appear in Crill or in Hess. In fact, 
Hess itself shows the unobviousness of this combination. Hess explains how difficult it 
would be to select items to view their images individually. Instead, Hess suggests 
forming a thumbnail of the original image of the item and including that thumbnail as 
part of the summary listing that is returned as search results. See column 3 lines 35-47 
of Hess. Hess says not one word about searching through the images. 

Hess also describes other searching, column 4 lines 28-38. The point of Hess is 
to return thumbnails as part of the search results. This is wholly different than the 
claimed technique of searching image databases. Hess describes in detail his different 
features including item maintenance (column 6), item registration (column 7), image 
harvesting (columns 7-9), item presentation (column 9), gallery presentation format and 
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found database access (columns 9-1 0). Nowhere is there any teaching or suggestion 
of the claimed feature that requires "using said image information to search said 
database associated with the server for items to be purchased which meet criteria 
specified in said image information". In fact, the recognition of the source of this 
problem is an important part of what is patentable in the present system, and there is 
not one word of the source of this problem in either Crill or Hess. 

Therefore, making the hypothetical combination of Crill in view of Hess would 
simply teach a Crill type image system, along with Hess's teaching that one should 
return image thumbnails as part of the search. Neither of these items of prior art 
recognizes the problem of how to find something that cannot be easily discussed in 
textual form. Since there is no teaching or suggestion of this problem, there is certainly 
no teaching or suggesting to modify Crill to "search the database for items to be 
purchased which meet criteria specified in said image information" quoting from claim 1 . 

Even if, hypothetical^, Hess did teach such a system, the idea of using Grill's 
techniques with product sales feature is entirely based on the teaching of the present 
specification, and the present specification's identification of a problem. Claim 1 
defines image information and searching a database using the image information for 
items to be purchased. This source of this problem: that a database of image item 
information could be searched for items to be purchased, is in no way taught or 
suggested by the cited prior art. As described in the specification, this produces 
significant advantages including ways to find items even when a prospective purchaser 
does not know the right words to describe those items. Nothing in the prior art 
discloses this problem, and the solution to this problem that is posed by the Patent 
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Office's hypothetical combination of prior art is quite simply based entirely on hindsight. 

Again, Crill says not one word about using his image searching feature for "items 
to be purchased which meet criteria specified in said image information". 

Hess teaches not one word about using any image searching feature for to find 
items to be purchased. Neither reference suggests the source of this problem. Since 
neither reference suggests it, the hypothetical combination certainly cannot teach it. 
The only thing that teaches the source of this problem is the present specification. 

Therefore, to summarize, this subject matter is completely unsuggested by the 
cited prior art, the rejection is wholly based on the teaching of the present specification, 
and does not properly rely on the scope and contents of the cited prior art. Accordingly, 
the Patent Office has not met their burden of providing a prima facie showing of 
unpatentability. Moreover, the claimed technique has the advantage of finding items 
that cannot easily be described in any other way, or more easily finding items when 
there are multiple words that can be used to find such an item. 

Consider the advantage of the present system. Claim 1 can be used, for 
example, to allow user to obtain an image of something they want to purchase without 
knowing the name of that thing. Claim 1 can be used to allow a purchaser to obtain an 
image of something they want to buy, even when there are many names for that thing. 
That image can be used to search the database for items which look like that image. In 
this way, a user may be able to find items on a website in a way that is not been 
possibly previously possible. This application is possible using the system of claim 1 . 
No recognition of this problem for the source of the problem is taught by the cited prior 
art. For these reasons, the cited prior art, no matter how combined, does not fairly 



-10- 



ATTORNEY DOCKET NO. VisualDatabase/SCH 
Serial No.: 09/683,599 



suggest the limitations noted above from claim 1 . 

The dependent claims should be allowable for similar reasons. 

Claim 3 and other claims like claim 3 define exclusion information to exclude 
from the search results. The rejection attempts to read this on the "cropping" that is 
taught by Grill. However, the cropping taught by Crill takes information out of the 
image . This is entirely different than the words of the present claims which require that 
the image information (note that the image information is what is entered at the client in 
claim 1) "includes exclusion information to exclude from said search results, and said 
server forms said search results which do not include said exclusion information". 

Crill may crop the image, but the cropping of the image is not entering exclusion 
information to exclude from the search results, or entering ANYTHING, for that matter. 
Moreover, Crill does not enter the exclusion information as "one of said parameters 
associated with said image information...". A cropping is carried out in the image in 
Crill, not on the image information used to initiate the search as claimed, and certainly 
not as a parameter associated with the information used to carry out the search. This 
rejection simply IGNORES this important claim language. 

Claim 4 requires selection of a more important image portion as one of the 
parameters associated with the image information used for the search. Claim 4 was 
rejected over Crill in view of Hess and further in view of Jain. 

initially, it is noted that the rejection alleges that each of claims 1-20 are rejected 
based on Crill in view of Hess, but then on page 5 admits that "Crill and Hess do not 
explicitly teach said client forms search results which are weighted according to said 
more important image portion", that is, using the words of claim 4 and 12. With all due 



-11- 



ATTORNEY DOCKET NO. VisualDatabase/SCH 
Serial No.: 09/683,599 

respect, the rejection is logically inconsistent on this point. 

In any case, Jain teaches a system of classifying images, and determining which 
parts are similar. The rejection draws attention to column 18 line 63 through column 9 
line 1 1 of Jain. This portion discusses the compare between two images which 
computes "overall visual similarity for two given images". This creates a weighted 
overall score.. A heterogeneous compare compares schemas for images with similar 
primitives. A threshold compare finds a similarity threshold. Query optimization is 
described beginning at line 30, and describes how the different primitives may be used 
as an initial search comparison. However, these thresholds and primitives provide no 
teaching or suggestion of selecting a more important image portion , as claimed. The 
rejection also refers to Jain 's column 3 lines 30-40. This refers to a similar scoring 
based on similarity between different images. The score may be used to rank orders of 
the degree of similarity, but this has nothing to do with selecting a most important part 
of the image portion as one of the search parameters, as claimed. Jain teaches 
ranking the images based on the return search parameters, not selecting a more 
important image portion 

Claim 9 recites an electronic commerce site and price information associated 
with the search results. This further emphasizes the distinction over the cited prior art. 
The rejection only says that "these limitations are well-known in the art and would have 
been an obvious modification...". However, the prior art has not one word about 
returning search results from an image search, including price information. These 
features are not at all known in the art, and this claim is even further patentable over 
the cited prior art. 
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Claim 10 defines entering image information, sending that image information to a 
server and returning search results from the server including price information 
associated with the items in the search results. As described above, Crill in view of 
Hess teaches nothing about searching using image information and returning price 
results associated with items in the search result. See the detailed arguments above. 

The other dependent claims should be allowable for reasons above. Claim 12 
should be allowable for reasons discussed above with respect to claim 4, and claim 13 
should be allowable for reasons discussed above with respect to claim 3. 

Claim 15 defines that the image information which is used as part of the search 
includes "size information" so that the user can use the size of the items they are 
searching as one of the search items. The rejection draws attention to Grill's column 7 
lines 1 7-31 . This cited section, however, simply teaches that a user can draw a circle 
by first clicking on a drawing icon representing a circle and then clicking and dragging 
the computer mouse to change its size. There is not one word about entering size 
information, much less searching Grill's database or any other database based on such 
size information. That rejection, with all due respect, is wholly improper and incorrect. 

Claim 17 defines a computer that searches a database that includes price 
information and returns search results to match the searching information and price 
information. As noted above, this subject matter is in no way taught or suggested by 
the cited prior art, and the source of the problem is never taught or suggested by the 
cited prior art. The dependent claims which depend from claim 17 should be allowable 
for similar reasons. 

In response to arguments in the final official action, the Patent Office apparently 



-13- 



ATTORNEY DOCKET NO. VisualDatabase/SCH 
Serial No.: 09/683,599 

takes issue with applicant distinguishing over the prior art by discussing the prior art 
individually. It should be apparent that applicant is discussing the hypothetical 
combination of prior art. With all due respect, it is not understood how such a 
hypothetical combination of prior art could even be discussed without discussing the 
references individually. Each reference has to stand on its own. While the Patent 
Office certainly is entitled to make combinations of prior art when there is motivation to 
do so, there is no way to discuss these references without discussing them individually. 

For reasons that should hence be evident from the above, it is respectfully 
suggested that the rejection is in error and should be reversed. 

Please charge any unpaid fees due in connection with this response to Deposit 
Account No. 50-1387. 

Respectfully submitted, 



Date: _6/1 4/07 _/Scott C Harris/, 

Scott C. Harris 
Reg. No. 32,030 

Customer No. 23844 
Scott C. Harris, Esq. 
P.O. Box 927649 
San Diego, CA 92192 
Telephone: (619) 823-7778 
Facsimile: (858) 678-5082 
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CLAIMS APPENDIX 

1 . A system, comprising: 

a client which allows entry of image information; and 

a server, including a database associated with the server, said server connected 
to said client to receive said image information and using said image information to 
search said database associated with the server for items to be purchased which meet 
criteria specified in said image information and forming search results based on said 
image information. 

2. A system as in claim 1 ,wherein said client allows forming forming initial image 
information, and subsequently setting parameters associated with said initial image 
information using a user interface. 

3. A system as in claim 2, wherein one of said parameters associated with said 
image information includes exclusion information to exclude from said search results, 
and said server forms said search results which do not include said exclusion 
information. 

4. A system as in claim 2, wherein one of said parameters includes a selection 
of a more important image portion, which is more important than other image portions, 
and said client forms search results which are weighted according to said more 
important image portion. 
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5. A system as in claim 2, wherein one of said parameters includes an image 
size, and said client forms search results which only include results having said 
specified image size. 

6. A system as in claim 2, wherein said client includes a scanner to allow entry 
of said initial image information. 

7. A system as in claim 2, wherein said client includes a tablet to allow entry of 
said initial image information. 

8. A system as in claim 2, wherein one of said parameters associated with said 
image information includes a color, and said server operates to find items based on 
said color information in addition to said image information. 

9. A system as in claim 1 , wherein said server is associated with an electronic 
commerce site, and said search results include price information for items associated 
with said search results. 

10. A method, comprising: 

entering image information to a client on the network; and 
sending said image information to a server on said network and using said image 
information to search database information on said server on said network; and 
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returning search results from said server to said client including price information 
associated with items in said search results. 

1 1 . A method as in claim 1 0, wherein said entering image information comprises 
entering an initial image, and entering parameters associated with the search of the 
initial image. 

12. A method as in claim 10, wherein said entering image information comprises 
entering an initial image to be searched, and selecting a part of the initial image 
information which represents a more important part of the image information. 

13. A method as in claim 10, wherein said entering image information comprises 
entering an initial image to be searched, and entering exclusion information to be 
excluded from said search. 

14. A method as in claim 13, wherein said exclusion information includes image 
information. 

15. A method as in claim 10, wherein said entering image information includes 
entering an initial image, and entering size information associated with said initial 
image, wherein said search results are based on said size information. 
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1 6. A method as in claim 1 1 , further comprising displaying said image 
information as part of a graphical user interface, and using said graphical user interface 
to enter said parameters. 

17. A system, comprising: 

a computer, connected to a publicly available network, and including a database 
associated therewith, said database including image information for each of a plurality 
of items to be sold over said publicly available network, and price information for each 
of said items, said computer accepting searching image information over said publicly 
available network, using said image information to search said image information in 
said database, to return search results including items from said database which match 
said searching image information and price information for each of said items from said 
database which match said searching information. 

18. A system as in claim 17, wherein said searching image information includes 
an image, and additional information about the searching, in addition to said image. 

19. A system as in claim 18, wherein said additional information includes 
exclusion information indicative of excluded items to be excluded from said searching. 

20. A system as in claim 18, wherein said additional information includes color 
information indicative of the desired colors to be included in the searching. 
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EVIDENCE APPENDIX 
None 
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RELATED PROCEEDINGS APPENDIX 
None 
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